Papers with sentence embedding models
AugCSE: Contrastive Sentence Embedding with Diverse Augmentations (2022.aacl-main)
Copied to clipboard
| Challenge: | Similar work has shown that a single augmentation can be used to learn a robust generalpurpose representation with contrastive learning. |
| Approach: | They propose a unified framework to utilize diverse sets of data augmentations to achieve a better, general-purpose sentence embedding model. |
| Outcome: | The proposed framework achieves state-of-the-art results on downstream transfer tasks and performs competitively on semantic textual similarity tasks, using only unsupervised data. |
Capturing the Relationship Between Sentence Triplets for LLM and Human-Generated Texts to Enhance Sentence Embeddings (2024.findings-eacl)
Copied to clipboard
| Challenge: | Recent advances in building sentence embedding models have centered on replacing traditional human-generated text datasets with those generated by LLMs. |
| Approach: | They propose a loss function that incorporates Positive-Negative sample Augmentation within the contrastive learning objective to enhance sentence embeddings using both human and LLM-generated datasets. |
| Outcome: | The proposed model mitigates the sentence anisotropy problem in Wikipedia corpus and improves Spearman’s correlation in standard Semantic Textual Similarity (STS) tasks (+1.47% compared to CLHAIF). |
Beyond Fine-tuning: Few-Sample Sentence Embedding Transfer (2020.aacl-main)
Copied to clipboard
| Challenge: | Fine-tuning (FT) pre-trained sentence embedding models on small datasets has been shown to have limitations. |
| Approach: | They propose to combine embeddings from a pre-trained model with a simple sentence embeddable model. |
| Outcome: | The proposed approach outperforms FT on small datasets with negligible computational overhead. |
Leveraging Multi-lingual Positive Instances in Contrastive Learning to Improve Sentence Embedding (2024.eacl-long)
Copied to clipboard
| Challenge: | Recent trends in learning monolingual and multilingual sentence embeddings are based on contrastive learning (CL) among an anchor, one positive and multiple negative instances. |
| Approach: | They propose to leverage multiple positives to improve learning of multilingual sentence embeddings by using an anchor, one positive, and multiple negative instances. |
| Outcome: | The proposed approach improves retrieval, semantic similarity, and classification performance on unseen languages. |
Disentangling Semantics and Syntax in Sentence Embeddings with Pre-trained Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained language models have been successful on a wide range of NLP tasks . however, contextual representations from pre-trated models contain entangled semantic and syntactic information. |
| Approach: | They propose a semantic sentence embedding model that disentangles semantics and syntax from pre-trained models. |
| Outcome: | The proposed model outperforms state-of-the-art models on unsupervised semantic similarity tasks. |
PTEB: Towards Robust Text Embedding Evaluation via Stochastic Paraphrasing at Evaluation Time with LLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing evaluations of sentence embedding models rely on static tests like the Massive Text Embedding Benchmark (MTEB) repeated tuning on a fixed suite can inflate reported performance and obscure real-world robustness. |
| Approach: | They propose a dynamic protocol that generates meaning-preserving paraphrases at evaluation time and aggregates results across multiple runs. |
| Outcome: | The proposed protocol generates meaning-preserving paraphrases at evaluation time and aggregates results across multiple runs. |
Improved Universal Sentence Embeddings with Prompt-based Contrastive Learning and Energy-based Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing contrastive methods for learning universal sentence embeddings have limitations due to their over-parameterization and poor performance under domain shift settings. |
| Approach: | They propose to integrate an Energy-based Hinge loss to enhance the pairwise discriminative power of contrastive learning for sentence embeddings by combining PLMs with energy-based learning. |
| Outcome: | Empirical results show that the proposed method improves on seven standard semantic textual similarity tasks and a domain-shifted STS task. |
Syntax-guided Neural Module Distillation to Probe Compositionality in Sentence Embeddings (2023.eacl-main)
Copied to clipboard
| Challenge: | Past work on sentence embedding models faces issues determining the causal impact of implicit syntax representations. |
| Approach: | They construct a neural module net based on a transformer model and train it end-to-end to approximate the sentence’s embedding. |
| Outcome: | The proposed model captures whether syntax is a strong model of its compositional ability. |
Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing sentence embeddings models are monolingual, and only for English . a new method allows to create multilingual versions from monolingual models . |
| Approach: | They propose a method to extend existing sentence embedding models to new languages . they use a translated sentence to generate sentence embeds for the source language . |
| Outcome: | The proposed method improves accuracy for multilingual setups and languages. |
TR-MTEB: A Comprehensive Benchmark and Embedding Model Suite for Turkish Sentence Representations (2025.findings-emnlp)
Copied to clipboard
| Challenge: | TR-MTEB is the first large-scale, task-diverse benchmark for sentence embedding models for Turkish. |
| Approach: | a new benchmark evaluates sentence embedding models for Turkish . TR-MTEB covers six core tasks and 26 high-quality datasets . |
| Outcome: | The TR-MTEB benchmark covers six core tasks and includes 26 high-quality datasets . the models achieve competitive performance across most tasks and significantly improve on baseline models. |
MultiClaimNet: A Massively Multilingual Dataset of Fact-Checked Claim Clusters (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a growing number of unverified claims and expanding size of fact-checked databases require alternative, more efficient solutions. |
| Approach: | They propose to group fact-checked claims into multilingual clusters to improve claim retrieval and validation. |
| Outcome: | The proposed approach reduces redundancy by grouping claims into clusters . the proposed dataset contains 85.3K fact-checked claims written in 78 languages . |
StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding (2023.emnlp-main)
Copied to clipboard
Cheng Jiayang, Lin Qiu, Tsz Chan, Tianqing Fang, Weiqi Wang, Chunkit Chan, Dongyu Ru, Qipeng Guo, Hongming Zhang, Yangqiu Song, Yue Zhang, Zheng Zhang
| Challenge: | Analogy-making between narratives is crucial for human reasoning . despite its importance, there has been limited research on story analogies . |
| Approach: | They construct a large-scale story-level analogy corpus with 24K story pairs . they find that the tasks are incredibly difficult for large language models such as ChatGPT . |
| Outcome: | The proposed corpus contains 24K story pairs from diverse domains with human annotations on two similarities from the extended Structure-Mapping Theory. |
Bridging Continuous and Discrete Spaces: Interpretable Sentence Representation Learning via Compositional Operations (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to learn sentence embeddings do not capture the semantic similarity of sentences. |
| Approach: | They propose a framework that integrates compositional sentence operations into the embedding space and optimizes operator networks and a bottleneck encoder-decoder model to produce meaningful and interpretable sentence embeddables. |
| Outcome: | The proposed framework improves the interpretability of sentence embeddings on four textual generation tasks while maintaining strong performance on traditional semantic similarity tasks. |
When Claims Evolve: Evaluating and Enhancing the Robustness of Embedding Models Against Misinformation Edits (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing claims-matching systems that use sentence embedding models are not robust to edits as users interact with claims online. |
| Approach: | They propose a perturbation framework that generates valid and natural claim variations and evaluate different mitigation approaches to improve their findings. |
| Outcome: | The proposed framework evaluates embedding models in a multi-stage retrieval pipeline and identifies the effectiveness of mitigation approaches. |